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ABSTRACT 


This  thesis  uses  pre-commissioning  academic  and  demographic  factors,  along  with 
flight  school  performance  data  to  measure  pilot  success  in  flight  school.  The  goal  is  to 
determine  if  undergraduate  major  or  sch-  1  attended  affect  flight  school  performance. 
Measures  of  effectiveness  include:  (1)  Fli:  School  Completion  Status,  (2)  Aviation  Pre- 
Flight  Indoctrination  Composite  Scores,  a:  }  Primary  Flight  Training  Composite  Scores. 

Recruitment  for  naval  aviators  is  focused  individuals  with  “technical  majors,”  according 
to  present  policy  of  the  Naval  Recruiting  nmand.  This  recruiting  philosophy  is  based 
on  the  “Rickover  Hypothesis,”  which  postulates  that  naval  officers  with  technical  degrees 
are  superior  to  naval  officers  with  non-technical  degrees.  The  Logit  model  showed  that 
aviators  with  engineering  degrees  have  a  statistically  greater  chance  of  completing  flight 
school  than  aviators  with  non-engineering  technical  or  non-technical  degrees.  In  addition, 
the  results  showed  an  association  between  academic  background  and  flight  school 
performance.  This  research  justifies  the  current  Navy  policy  of  concentrating  aviator 
recruitment  efforts  on  individuals  with  technical  degrees. 
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EXECUTVE  SUMMARY 


The  Naval  Recruiting  Command  aviator  recruiting  policy  is  to  obtain  college 
graduates  who  possess  engineering  and  science  degrees.  This  recruiting  philosophy  is 
based  on  the  “Rickover  Hypothesis,”  which  postulates  that  naval  officers  with  technical 
degrees  are  superior  to  naval  officers  with  non-technical  degrees.  Bowman  (1990) 
examined  this  hypothesis  for  officers  undergoing  nuclear  training  and  found  that  technical 
degrees  significantly  increased  the  probai  ility  of  completing  this  training,  but  that  little 
effect  on  subsequent  performance  and  promotion  was  observed.  The  current  study 
examines  the  Rickover  hypothesis  for  student  aviators  by  comparing  their  academic  and 
flight  performance  during  training.  These  comparisons  are  made  based  on  flight  school 
completion/attrition  status  and  on  composite  scores  attained  during  Aviation  Pre-flight 
Indoctrination  and  Primary  flight  training.  It  is  expected  that  differences  in  performance 
will  be  found  based  upon  undergraduate  major  or  college  attended.  In  addition,  differences 
in  performance  will  be  noted  based  upon  scores  received  on  the  aviation  selection  test 
battery.  Individuals  who  have  a  higher  Academic  Qualification  Rating  and/or  a  higher 
Flight  Aptitude  Rating  will  perform  better  during  the  academic  and  flight  portions  of 
training. 

Using  Classification  and  Regression  Trees,  Logistic  regression  and  Least-Squares 
regression,  the  performance  of  student  aviators  in  the  aviation-training  pipeline  (measured 
by  completion  or  attrition,  and  by  flight  school  grades)  is  modeled.  Demographic  variables 
from  the  officer  master  file,  along  with  aviation  selection  test  battery  performance  data,  are 
used  as  predictors  to  measure  success  in  flight  school.  These  factors  include  sex,  race. 
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ethnicity  and  commissioning  source,  along  with  undergraduate  major,  college  attended,  and 
undergraduate  academic  performance  grades. 

Classification  Tree  and  Logit  models  are  developed  to  predict  individual  success  in 
completing  training.  Results  indicate  that  academic  major  is  a  significant  predictor  of 
flight  school  completion,  along  with  college,  race,  Aviation  Qualification  Rating  and  Flight 
Aptitude  Rating.  Engineers  are  more  likely .l-.  complete  flight  school  than  individuals  who 
hold  non-engineering  technical  or  non-technical  degrees.  Further,  graduates  from  the 
Naval  Academy  have  a  significantly  higher  completion  rate  than  individuals  who  attended 
other  colleges. 

Regression  Tree  and  Least-Squares  regression  models  are  developed  to  measure 
relative  success  of  individuals  throughout  the  flight  training  pipeline  by  using  predictors  to 
model  their  academic  and  flight  performance  grades  during  Aviation  Pre-flight 
Indoctrination  and  Primary  flight  training.  Only  individuals  who  successfully  completed 
flight  school  are  considered.  Significant  predictor  variables  include  Academic 
Qualification  Rating,  Flight  Aptitude  Rating,  race,  ethnicity,  college  and  major,  indicating 
that  each  predictor  has  an  influence  on  the  composite  score  attained  while  attending  flight 
school. 

In  summary,  there  is  evidence  to  suggest  that  academic  major  and  college  attended 
affect  performance  in  flight  school.  These  factors,  along  with  race,  ethnicity.  Academic 
Qualification  Rating  and  Flight  Aptitude  Rating  can  be  used  by  the  recruiting  command  to 
screen  potential  applicants  using  the  models  developed.  Individuals  can  be  compared, 
based  on  their  individual  characteristics,  to  determine  the  best  candidates  for  acceptance 
into  flight  school. 
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I.  INTRODUCTION 

A.  BACKGROUND 

As  the  United  States  makes  the  transition  into  the  21st  century,  changes  in  the 
recruiting  and  selection  process  for  naval  aviators  could  bring  an  improvement  to  the 
Navy's  return  on  investment.  The  qualification  process  for  aviators  is  intense,  with  many 
applicants  competing  for  a  limited  number  of  billets.  Many  factors  contribute  to  selection 
failure.  First  among  them  are  stringent  initial  screening  requirements.  The  applicant 
screening  process  requires  all  individuals  to  pass  both  an  Aviation  Selection  Test  Battery 
(ASTB)  and  a  military  physical  exam.  These  initial  qualification  standards  result  in  a 
disqualification  rate  of  75%  from  the  initial  applicant  pool  (Williams,  Albert,  and  Blower, 
1 999).  Further,  individuals  must  then  apply  for  acceptance  into  the  flight-training  program, 
whereupon  successful  candidates  are  ordered  to  flight  school  for  initial  and  follow-on  flight 
training.  Each  stage  in  the  qualification  process  brings  about  a  reduction  in  the  pool  of 
potential  aviators,  resulting  in  1  fleet-qualified  aviator  for  about  every  25  applicants  (Wahl, 
1998). 

The  aviator  recruiting  effort  is  focused  on  individuals  with  “technical  majors,” 
according  to  present  policy  of  the  Naval  Recruiting  Command  (NRC,  1999).  This  idea  is 
in  line  with  the  “Rickover  Hypothesis,”  which  presumes  that  individuals  with  technical 
degrees  are  better  prepared  and  make  better  naval  officers  than  individuals  with  non¬ 
technical  degrees.  However,  Bowman  (1990)  found  a  very  weak  statistical  relationship 
between  USNA  major  and  fleet  performance,  using  fitness  reports  and  other  job 
performance  variables.  In  addition,  he  argued  that  the  need  for  technical  skills  diminishes 
as  officers  advance  to  positions  requiring  greater  managerial  and  administrative  skills 
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(Bowman,  1990).  Bowman  found  that  although  a  technical  degree  significantly  affects  the 
likelihood  of  completing  the  nuclear  training  pipeline,  little  effect  on  performance  and 
promotion  is  apparent  once  individuals  complete  training  and  enter  the  fleet  (Bowman, 
1990).  Performance  in  areas  not  directly  associated  with  the  formal  training  process,  such 
as  leadership,  management,  and  interpersonal  skills,  affect  overall  individual  performance 
and  promotion  rates,  yet  have  little  to  do  with  academic  background.  Thus,  by  limiting  the 
potential  applicant  pool  to  individuals  with  technical  degrees,  a  significant  portion  of  the 
population  of  potential  aviators  is  removed  from  consideration.  These  are  individuals  who 
lack  a  technical  degree  but  might  otherwise  pass  screening  requirements,  flight  school  and 
thus,  become  Naval  aviators. 

B.  RESEARCH  OBJECTIVES 

This  study  examines  the  effect  of  academic  background  on  pilot  performance 
within  the  flight-training  pipeline.  Each  analysis  focuses  on  the  effect  of  academic 
achievement  as  measured  by  grades,  undergraduate  major,  and  the  correlation  between 
college  quality,  commissioning  source  and  flight  school  performance.  Comparison 
between  technical  and  non-technical  aviators  is  made  to  determine  if  pilots  with  technical 
degrees  perform  better  through  the  flight-training  pipeline. 

A  second  objective  was  to  analyze  the  effect  of  undergraduate  major  on  retention 
and  promotion  rates  of  pilots  who  completed  the  flight-training  pipeline  during  the  1980’s, 
based  upon  their  flight  school  performance,  and  in  conjunction  with  other  demographic 
variables.  This,  however,  was  impossible  because  of  incomplete  data  on  flight  school 
aviator  performance  in  years  previous  to  1990. 
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c. 


SCOPE  AND  LIMITATIONS 


The  focus  of  this  study  is  on  the  effects  of  undergraduate  academic  major  and 
college  attended  on  student  aviator  performance.  Effects  of  other  college  characteristics, 
such  as  participation  in  athletics  or  other  extracurricular  activities,  are  not  included.  This 
intention  is  to  test  the  validity  of  the  “Rickover  Hypothesis”  with  regard  to  the  aviation 
community  and  to  provide  recruiters  wit:  information  that  will  allow  them  to  focus  on 
those  college  students  who  are  likely  perform  well  in  the  aviation  community. 
Academic  major  and  college  are  included  i  the  models  as  independent  variables.  Flight 
Aptitude  Rating  (FAR)  and  Academic  Qualification  Rating  (AQR)  are  also  included  since 
they  are  designed  to  predict  performance  in  flight  school.  Demographic  variables  such  as 
sex,  race,  ethnicity  and  commissioning  source  are  included  to  control  for  possible 
confounding.  (For  example,  if  it  were  the  case  that  black  aviators  both  performed  better  on 
average  in  flight  school  and  were  more  likely  to  have  technical  degrees  than  non-blacks, 
the  effect  of  having  a  technical  degree  would  be  over-stated  in  a  model  that  did  not  include 
race  as  an  independent  variable.) 

e 

This  study  is  limited  to  pilots  who  graduated  from  flight  school  between  1990  and 
1999.  Individuals  from  the  United  States  Naval  Academy  (USNA),  Naval  Reserve  Officer 
Training  Units  (NROTC)  or  from  Officer  Candidate  School  (OCS)  are  included.  The  data 
set  does  not  include  those  who  transferred  into  the  aviation  community  from  another 
community  or  from  another  service.  Officers  with  prior  enlisted  service  are  omitted  if  they 
served  greater  than  four  years  in  an  enlisted  capacity. 

The  most  recent  revision  to  the  ASTB  occurred  in  1992.  Due  to  the  changeover 
(further  described  in  Chapter  II),  it  is  not  yet  possible  to  consider  the  long-term  retention 
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and  promotion  characteristics  of  officers  who  have  qualified  under  the  present  applicant 
screening  guidelines.  Because  it  takes  up  to  two  years  from  initial  test  taking  to 
commissioning,  and  a  further  two  years  for  individuals  to  complete  their  flight  training, 
these  officers  could  only  recently  have  started  their  obligated  service  requirement  and 
would  not  have  yet  reached  time-in-grade  requirements  for  Lieutenant  Commander.  In 
fact,  most  will  not  have  yet  completed  their  initial  service  requirement.  Therefore,  it  is 
difficult  to  predict  retention  likelihood  for  these  individuals  from  results  found  in  this 
study. 

D.  ORGANIZATION 

This  study  is  organized  into  five  chapters.  Following  the  introduction  and 
background  contained  in  Chapter  I,  Chapter  II  reviews  previous  studies  and  literature 
related  to  this  area  of  research.  Chapter  III  describes  the  data  files  that  are  employed  for 
this  research.  A  detailed  description  of  model  specifications  and  an  overview  of 
methodology  are  given,  with  an  explanation  of  the  dependent  and  independent  variables. 
Chapter  IV  presents  the  results  from  this  study  using  Classification  and  Regression  Trees 
(CART),  Logistic  regression  and  Least-Squares  regression.  Chapter  V  offers  conclusions 
and  recommendations  based  on  the  results  of  the  previous  chapter. 
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II.  LITERATURE  REVIEW 


A.  QUALITIES  OF  AN  AVIATOR 

Many  studies  have  been  conducted  that  examine  the  characteristics  of  “good” 
aviators;  however  most  have  focused  on  the  applicant  selection  process,  rather  than 
measuring  skill  and  performance  over  the  long  term.  According  to  the  Federal  Aviation 
Administration,  no  one  is  bom  a  nati  al  pilot.  Competent  commercial  pilots  become 
so  through  study,  hard  work  and  experit  ce”  (U.S.  Department  of  Transportation,  p.  1, 
1995).  In  order  to  fly  safely,  all  aviato;  .,  must  master  basic  airmanship,  operation  of 
aircraft  systems,  and  navigation.  Commercial  pilot  tests  cover  a  wide  range  of  subject 
areas,  in  addition  to  specific  instruction  in  the  aircraft  category  for  which  rating  is  sought. 
Military  aviators  must  also  master  these  functional  areas,  in  addition  to  specialized 
requirements  necessary  to  work  in  a  combat  role.  They  must  also  understand  the  operation 
of  weapon  systems  and  cope  with  high  stresses  imposed  by  operation  in  combat 
environments.  A  list  of  knowledge  areas  from  the  1995  Commercial  Pilot  Knowledge  Test 
Guide  is  given  below  as  suggestions  of  topics  to  be  covered  by  rating  exams  (U.  S. 
Department  of  Transportation,  1995): 

1.  The  Federal  Aviation  Regulations  that  apply  to  commercial  pilot  privileges, 
limitations,  and  flight  operations. 

2.  Accident  reporting  requirements  of  the  National  Transportation  Safety  Board. 

3.  Basic  aerodynamics  and  the  principles  of  flight. 

4.  Meteorology,  including  recognition  of  critical  weather  situations,  wind  shear 
recognition  and  avoidance,  and  the  use  of  aeronautical  weather  reports  and 
forecasts. 

5.  The  safe  and  efficient  operation  of  aircraft. 
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6.  Weight  and  balance  computation. 

7.  Use  of  performance  charts. 

8.  Significance  and  effects  of  exceeding  aircraft  performance  limitations. 

9.  Use  of  aeronautical  charts  and  magnetic  compass  for  pilotage  and  dead 
reckoning. 

1 0.  Use  of  air  navigation  facilities. 

1 1 .  Aeronautical  decision-making  and  judgment. 

12.  Principles  and  functions  of  aircraft  systems. 

t 

13.  Maneuvers,  procedures,  and  emergency  operations  appropriate  to  the  aircraft. 

14.  Night  and  high-altitude  operations. 

15.  Descriptions  of  and  procedures  for  operating  within  the  National  Airspace 
System. 

In  addition  to  the  above  areas,  naval  aviators  must  also  master  the  complexities  of 
combat  weapon  systems  and  the  effects  each  has  when  carried  and  launched.  They  must 
understand  launch  parameters  of  the  weapons,  their  flight  characteristics,  and  the  additional 
system  controls  they  require.  In  another  area,  most  naval  aviators  are  faced  with  the 
obstacle  of  landing  an  aircraft  on  an  unstable  platform  (ship  or  aircraft  carrier),  limited  in 
size,  and  moving  relative  to  the  aircraft.  These  additional  factors  are  summed  up  by 
Pohlman  &  Fletcher  (1999),  who  state  that  military  pilots  must  (Pohlman  &  Fletcher, 
1999): 

1 .  Plan  the  route  through  space  in  relation  to  the  intended  target,  suspected  threats, 
actual  threats,  other  known  aircraft,  wingmen,  and  weapons. 

2.  Monitor  the  aircraft  display  for  electronic  notification  of  threats. 

3.  Differentiate  among  threat  displays  (these  can  portray  15  or  more  different 
threats). 
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4.  Plan  ingress  to  and  egress  from  the  target. 

5.  Set  switches  for  specific  missions  during  specific  periods  of  the  flight. 

6.  Monitor  radio  chatter  on  multiple  frequencies  for  new  orders  and  threat 
notification. 

7.  Monitor  progress  along  the  planned  route. 

8.  Calculate  course,  altitude,  and  •  ;rspeed  corrections. 

9.  Plan  evasive  maneuvers  for  ea  type  of  threat  and  position  during  the  mission. 

10.  Plan  weapons  delivery. 

In  short,  the  demands  of  military  (and  commercial)  flying  require  the  ability  to 
make  quick  mental  adjustments  using  proper  judgment  in  response  to  rapidly  changing 
situations.  Because  of  this  necessity,  assessment  procedures  to  determine  qualified 
candidates  for  aviator  training  are  rigorous.  Proper  candidate  selection  saves  time,  material 
and  funding,  and  results  in  improved  quality  and  operational  readiness.  Because  of  the 
expense  and  complexity  required  to  train  competent  pilots,  “almost  every  test  in  the 
psychological  arsenal  has  been  evaluated  at  one  time  or  another  to  determine  its 
applicability  for  aircrew  selection”  (Hunter,  p.  129,  1989).  In  addition,  Hilton  and  Dolgin 
wrote  that  there  may  be  no  other  “occupation  in  the  world  that  benefits  more  from 
personnel  selection  technology  than  that  of  military  pilot”  (Hilton  &  Dolgin,  p.  81, 1991). 

According  to  Pohlman  and  Fletcher  (1999),  three  conclusions  may  be  drawn  from 
review  of  studies  related  to  aviator  selection.  The  first  is  that  nearly  all  validation  studies 
conducted  concern  the  ability  to  predict  performance  in  training,  rather  than  long-term  pilot 
performance.  Inherent  in  this  reasoning  is  the  belief  that  an  aviator  who  does  well  in 
training  will  continue  with  good  performance  throughout  his  or  her  career.  Training 
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validation  is  good  because  it  identifies  potential  failures  early,  resulting  in  decreased  costs 
borne  by  the  government  or  individual  trainee.  According  to  Hunter,  flight  training  is  “the 
most  expensive  of  the  many  training  programs  conducted  by  the  military  services”  (Hunter, 
p.  129,  1989).  The  second  conclusion  is  that  there  is  little  relationship  between  general 
intelligence  and  aviator  performance.  It  was  noted  that  newer  tests  of  mental  ability  might 
better  identify  aspects  of  general  intelligence  that  predict  aviator  performance.  Finally, 
Hunter  (1989)  found  that  of  the  36  studies  conducted,  “only  those  concerned  with 
instrument  comprehension  and  mechanical  comprehension  were  consistent  predictors  of 
success.”  In  a  more  recent  study  by  Hunter  and  Burke  (1995),  it  was  found  that  the  best 
correlates  of  success  in  training  were  sample  tests  of  job  performance,  gross  dexterity, 
mechanical  understanding,  and  reaction  time.  General  ability,  quantitative  ability,  and 
education  were  again  found  to  be  poor  correlates  of  success  (Hunter  &  Burke,  1995). 

B.  NAVAL  AVIATION  SELECTION  TESTS 

Predictors  of  success  in  flight  training  were  developed  during  the  Second  World 
War.  Large  numbers  of  naval  aviators  were  needed  to  meet  the  needs  of  the  United  States 
war  effort,  and  a  selection  process  for  potential  candidates  was  introduced.  Subsequent 
revision  and  examination  led  to  the  formulation  of  the  first  naval  aviation  selection  test, 
called  the  Aviation  Selection  Test  Battery  (ASTB)  (Dean,  1996).  This  test,  revised  in 
1953,  and  again  in  1971,  was  composed  of  two  parts:  an  Academic  Qualification  Test 
(AQT)  and  a  Flight  Aptitude  Rating  (FAR).  The  test  was  used  as  a  measure  to  determine 
potential  applicant  success  in  flight  training,  using  biographical  data  derived  from  a 
questionnaire  that  asked  about  family  background,  personal  and  medical  history, 
environmental  influences,  education  and  vocational  interests,  in  addition  to  academic  skills. 
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The  AQT  portion  was  a  general  intelligence  instrument  designed  to  predict  performance  in 
the  academic  phase  of  training.  The  FAR  was  a  composite  score  based  upon  individual 
scores  of  a  Mechanical  Comprehension  Test  (MCT),  a  Biographical  Inventory  (BI)  and  a 
Spatial  Apperception  Test  (SAT).  This  composite  score  was  intended  to  predict  the 
probability  of  a  student’s  success  in  the  flight  portion  of  training.  In  1992,  the  test  was 
again  replaced  in  favor  of  a  newer  version. 

The  1992  ASTB  is  the  most  recent  revision  in  a  series  of  cognitive  tests  used  as  a 
selection  measure  for  potential  naval  aviators.  This  revision  took  place  due  to  demographic 
changes,  decreases  in  predictive  validity  of  the  previous  test,  possible  AQT/FAR  test 
compromise,  and  finally,  due  to  changes  in  federal  guidelines  regarding  employee  selection 
procedures  (Frank  &  Baisden,  1993).  The  new  version  of  the  ASTB  consists  of  six  sub¬ 
tests:  1)  Math-Verbal  Test  (MVT),  which  tests  general  intelligence;  2)  Mechanical 
Comprehension  Test  (MCT)  which  test  the  ability  to  perceive  physical  relationships  and 
solve  practical  problems  in  mechanics;  3)  Spatial  Apperception  Test  (SAT),  which  tests  the 
ability  to  perceive  spatial  relationships  from  different  orientations;  4)  Aviation  and 
Nautical  Test  (AN),  which  tests  for  general  aviation  and  nautical  knowledge;  5) 
Biographical  Inventory  (BI)  which  is  a  questionnaire  of  personal  history  and  interests;  and 
6)  Aviation  Interest  (AI)  which  is  a  questionnaire  of  aviation-related  items.  Weighted 
combinations  of  the  sub-tests  result  in  three  separate  scores,  each  on  a  9-point  scale.  These 
scores  are  used  to  predict  attrition,  academic  performance,  and  basic  flight  performance  of 
potential  aviator  candidates.  The  first  score,  called  the  Academic  Qualification  Rating 
(AQR),  predicts  flight  school  academic  performance.  The  Flight  Aptitude  Rating  (FAR) 
predicts  basic  flight  performance  while  the  Biographical  Inventory  (BI)  predicts  attrition. 
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The  new  version  of  the  ASTB  differs  from  the  previous  tests  (utilizing  AQT/FAR  scores) 
in  that  only  the  BI  is  intended  to  predict  attrition  from  primary  flight  training.  Navy 
applicants  must  obtain  qualifying  scores  of  3,  4  and  4  on  the  AQR,  FAR,  and  BI  for  initial 
consideration  into  the  flight  program.  Candidates  receiving  qualifying  scores  are  further 
screened  prior  to  acceptance  into  the  training  pipeline. 

C.  THE  NAVAL  FLIGHT  TRAINING  PROGRAM 

Candidates  selected  for  naval  aviation  training  first  attend  a  six-week  course  of 
ground  school  training  at  Aviation  Preflight  Indoctrination  (API).  During  API,  students 
master  topics  such  as  aerodynamics,  aircraft  engines,  air  navigation,  meteorology,  flight 
rules  and  regulations  (FR&R),  physical  conditioning,  and  water  survival,  resulting  in  a  total 
of  23 1  hours  of  instruction  being  received  by  each  candidate.  Upon  completion  of  API, 
candidates  are  split  into  pilot  and  Naval  Flight  Officer  (NFO)  programs,  and  proceed  to 
their  respective  training  pipelines  (Williams,  Albert  &  Blower,  1999).  Primary  flight 
training  introduces  the  student  to  actual  flying  experience,  including  basic  instrument  and 
radio  instrument  familiarization,  close  formation  and  night  flying  exercises.  All  flights  are 
flown  either  in  actual  T-34C  Turbo  Mentor  aircraft  or  the  T-34C  flight  simulator.  Student 
aviators  are  graded  on  preflight  knowledge,  emergency  procedures,  ability  to  think  and 
react  under  stress,  and  other  items  related  to  the  particular  flight  mission.  Each  student 
receives  a  grade  between  1.0,  which  is  considered  “unsatisfactory,”  and  4.0,  considered 
“above  average,”  respectively.  There  are  530  graded  items  completed  by  each  student 
aviator.  A  final  score  ranging  from  1 .0  to  4.0  is  assigned,  based  upon  the  average  of  all 
scores  received.  After  successful  completion  of  primary  flight  training,  each  student  then 
enters  one  of  four  aircraft  pipelines  for  intermediate  flight  training.  Selection  to  a 
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particular  pipeline  is  based  on  flight  grades  and  the  current  needs  of  the  Navy.  In  general, 
however,  those  with  highest  grades  are  selected  for  jets,  followed  by  carrier-based  propeller 
aircraft,  maritime  propeller  aircraft  and  helicopters  (Wahl,  1998). 

Naval  aviators  are  disqualified  or  dropped  from  the  training  pipeline  for  a  variety  of 
reasons.  As  stated  above,  many  potential  applicants  are  disqualified  during  the  initial 
recruitment  and  selection  phase;  however,  significant  numbers  drop  out  during  the  training 
process.  A  large  cost  is  incurred  when  i  dividuals  drop  from  the  training  pipeline.  As 
applicants  progress  through  the  training  process,  the  cost  for  each  applicant  increases 
significantly.  Reasons  for  withdrawal  from  training  vary  significantly,  with  categories 
including  Drop  on  Request  (DOR),  Flight  Failure,  Not  Physically  Qualified  (NPQ),  Not 
Officer  Material  (NOM),  Not  Aeronautically  Adaptable  (NAA),  Academic  Failure  and 
Other  (misconduct,  etc).  Approximately  60%  of  all  candidates  fail  due  to  medical 
disqualification.  The  Navy  considers  this  amount  to  be  unavoidable  and  not  preventable. 
The  goal  for  the  Navy  is  to  limit  the  number  applicants  withdrawing  from  training  due  to 
academic,  flight  failure,  or  DOR.  Identification  and  early  intervention  by  the  Navy  may 
result  in  subsequent  cost  savings  to  the  training  command  and  the  Navy  as  a  whole. 

D.  OFFICER  PERFORMANCE  MEASUREMENT 

In  1976,  ADM  Hyman  G.  Rickover,  Director  of  Naval  Reactors,  testified  before  the 
House  Armed  Services  Committee  that  teaching  Management  as  a  major  subject  for  an 
undergraduate  did  not  contribute  to  the  ability  of  a  junior  officer  to  do  his  job.  He  believed 
that  all  midshipmen  should  be  taught  electives  limited  to  the  technical  sciences  and  that 
social  sciences  should  be  specifically  excluded  (Hearings  on  Military  Posture,  1976).  This 
belief  led  to  the  “Rickover  Hypothesis,”  that  the  best  naval  officers  are  those  who  have  a 
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technical  undergraduate  major  (Bowman,  1990).  Bowman  studied  USNA  graduates  from 
1976  through  1980  who  entered  the  Surface  and  Submarine  officer  communities.  He  found 
that  technical  expertise  diminished  as  officers  advance  to  positions  requiring  greater 
managerial  and  administrative  skills  (Bowman,  1990).  In  addition,  he  found  that  retention 
factors  are  based  upon  personal  characteristics,  including  perceived  monetary  options  near 
the  end  of  one’s  obligation,  rather  than  academic  background.  Bowman’s  study  applied 
only  to  USNA  graduates,  rather  than  the  whole  population  of  officers  in  the  communities 
he  studied. 

Other  studies  that  measure  naval  officer  performance  in  a  wide  range  of  settings 
have  been  conducted.  These  examine  topics  ranging  from  specific  aviator  primary  flight 
performance  of  USNA  graduates  (Reinhart,  1998),  to  Surface  Warfare  Officer  (SWO) 
performance  as  related  to  commissioning  source,  undergraduate  education  and  Navy 
training  (Nolan,  1993).  The  following  paragraphs  highlight  the  research  and  results  in 
areas  related  to  officer  productivity. 

Reinhart  (1998)  utilized  ASTB  scores,  along  with  other  demographic  variables,  to 
compare  primary  flight  training  performance  of  1995  and  1996  USNA  graduates.  He 
found  that  individuals  who  scored  higher  on  the  BI  were  more  likely  to  complete  primary 
flight  training  than  those  with  lower  scores.  In  addition,  he  found  that  those  with  higher 
AQR  and  FAR  scores  achieved  higher  grades  than  individuals  with  lower  scores. 

Foster  (1990)  measured  the  relative  productivity  of  SWO  and  Submarine  Officer 
(SO)  officers  from  different  accession  sources,  using  a  performance  index  derived  from 
aspects  of  officer  fitness  reports,  which  allowed  officers  to  be  ranked  or  compared  with  one 
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overall  grade.  Foster  found  that  USNA  graduates  were  rated  higher,  and  that  by  a  small 
margin,  they  were  promoted  earlier  than  graduates  of  other  commissioning  sources. 

Nolan  (1993)  conducted  a  study  of  SWO  promotion,  retention  and  qualifications,  as 
related  to  commissioning  source,  education  and  training.  He  used  “Barron’s  Profile  of 
American  Colleges”  to  rank  colleges  according  to  their  competitiveness.  In  addition,  he 
utilized  the  Academic  Profile  Code  (APC)  that  is  assigned  to  each  officer  and  which 
summarizes  portions  of  an  individual’s  undergraduate  academic  performance.  The  APC  is 
broken  down  into  three  sections:  grade  point  average  (GPA);  mathematics  qualification 
code  (MQC);  and  the  technical  qualification  code  (TQC).  His  results  suggest  that 
attendance  at  higher-rated  colleges,  having  higher  educational  quality,  was  positively 
correlated  to  higher  performance  measures  of  effectiveness,  such  as  promotion  and  early 
attainment  of  qualifications. 

Woelper  (1998)  measured  the  impact  of  college  grades,  undergraduate  major, 
college  quality  and  commissioning  source  on  SO  job  performance,  as  evidenced  by  early 
promotion  and  retention.  His  intent  again  was  to  test  the  “Rickover  Hypothesis”  that  a 
strong  technical  background  makes  better  naval  officers.  He  found  that  engineering  majors 
have  higher  completion  rates  through  the  training  pipeline;  however,  major  had  an 
insignificant  effect  on  junior  officer  performance  or  promotion  to  LCDR  (Woelper,  1998). 
E.  BACKGROUND  CONCLUSIONS 

Aviator  performance  is  difficult  to  measure.  However,  much  work  has  been  done 
to  identify  individual  traits  that  point  to  successful  completion  of  flight  training.  The 
civilian  literature  cited  above  (Puhlman  &  Fletcher,  1999)  indicates  that  primary  in 
importance  is  an  individual’s  ability  to  master  the  mechanics  and  coordination  required  for 
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flying.  The  ASTB  enables  the  U.  S.  Navy  to  screen  potential  applicants,  prior  to 
committing  limited  funding  and  resources  to  individuals  not  suited  for  aviation.  Further, 
Aviation  Pre-flight  Indoctrination  attempts  to  identify  as  early  as  possible  those  individuals 
who  are  unable  to  complete  the  training  regimen.  Research  conducted  on  the  SO 
community  seems  to  indicate  that  individuals  with  technical  degrees  are  better  than  those 
with  non-technical  degrees,  at  least  during  training  (Woelper,  1998).  Work  has  been 
conducted  that  indicates  that  while  academic  background  has  a  positive  influence  on  officer 
performance  at  the  initial  training  point,  the  lasting  effect  of  undergraduate  experience 
diminishes  as  time  passes  (Woelper,  1998).  From  the  previous  literature  reviewed,  it  is 
expected  that  an  aviator’s  academic  background  or  college  attended  will  influence  flight 
school  performance.  In  addition,  higher  ASTB  scores  will  correlate  with  superior  flight 
school  performance,  as  evidenced  by  previous  studies  that  have  validated  this  test 
(Reinhart,  1998,  Williams,  Albert  &  Blower,  1999). 
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III.  DATA  AND  METHODOLOGY 


A.  DATABASE  DESCRIPTION 

The  first  data  set,  provided  by  the  Defense  Manpower  Data  Center  (DMDC),  was 
derived  from  the  Officer  Master  File  (OMF),  and  contained  both  pre-commissioning  and 
post-commissioning  characteristics  of  naval  officers.  Data  include  demographic, 
educational  background,  billet  assignment  and  promotion  factors.  The  second  data  set 
contained  information  on  flight  school  selection  and  performance  by  each  individual.  This 
data  set  Was  obtained  from  the  Naval  Aerospace  Medical  Research  Laboratory  (NAMRL) 
and  contained  8,882  observations  of  students  who  were  admitted  for  training  at  Naval 
Aviation  Schools  Command  in  Pensacola  from  1988  through  1999.  Included  in  this  set 
were  officers  from  other  service  branches  and  foreign  countries.  All  non-U SN  naval 
aviators  were  removed  from  the  data  set,  reducing  the  overall  number  to  6115  members. 
This  file  was  then  merged  with  the  OMF  file,  resulting  in  5123  matches  in  the  combined 
data  set.  The  resulting  aviator  subset  of  data  was  further  reduced  in  size  to  3937 
observations  due  to  missing  observations  of  other  key  variables,  including  undergraduate 
major,  aviation  pipeline,  flight  school  performance  scores  and  university  attended.  This 
data  set  was  further  divided  by  removing  the  NFO’s  from  consideration,  leaving  2612 
student  aviators.  Once  the  final  data  set  was  completed,  variables  were  created  or  factors 
modified  to  better  isolate  characteristics  of  interest  to  this  study.  A  breakdown  of  the  final 
data  set  column  descriptions  can  be  found  in  Appendix  A. 

B.  KEY  INDEPENDENT  VARIABLE  DESCRIPTION 

Several  key  variables  were  created  from  the  above  data  sets.  The  first  variable 
derived  was  “MAJOR,”  which  separated  college  degrees  awarded  into  three  categories. 
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These  categories  were  obtained  from  the  Office  of  the  Chief  of  Naval  Operations  (CNO) 
which  listed  “Non-Technical,”  “Technical”  and  “Math-Science  Technical”  majors  as 
possible  courses  of  study  for  the  future  naval  officer  (CNO,  1999).  Non-technical  majors 
such  as  history  and  accounting  were  assigned  to  the  first  factor  level,  while  non¬ 
engineering  technical  majors  such  as  math  and  physics  were  assigned  to  the  second  level. 
Engineers  were  assigned  to  the  third  factor  level.  As  indicated  above,  if  information  about 
the  college  degree  awarded  to  an  individual  was  absent  from  the  data  set,  then  the 
individual  was  removed  from  consideration. 

The  second  key  variable  is  “COLLEGE,”  which  was  created  using  a  combined 
listing  of  collegiate  rankings  as  provided  by  U.S.  News  and  World  Report  (Elfin,  1990; 
1995).  Colleges  were  rated  according  to  six  attributes  -  reputation,  selectivity,  faculty 
resources  and  financial  resources,  retention  and  alumni  satisfaction.  The  schools  were 
divided  into  “National”  and  “Liberal  Arts”  colleges.  They  were  then  ranked  according  to 
the  attributes  listed  above.  The  top  quartile  schools  by  rank  were  included  for  the  purposes 
of  this  study.  In  order  to  do  this,  the  top  schools  from  both  1990  and  1995  were  merged  to 
provide  coverage  for  aviators  who  otherwise  might  not  have  been  included.  Each 
individual  was  coded  as  having  attended  either  “none,”  “national”  or  “liberal”  to  signify  the 
school  rank  they  attended.  After  final  review,  the  factor  levels  for  “liberal”  and  “national” 
were  combined  because  only  a  few  individuals  attended  top  ranked  liberal  arts  colleges 
included  in  the  data  set.  None  of  the  service  academies  were  included  in  the  final  rankings 
by  U.S.  News  &  World  Report  even  though  a  significant  percentage  of  individuals  attended 
the  Naval  Academy.  These  individuals  were  placed  within  their  own  factor  level. 
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“Academic  Profile  code”  (APC)  is  another  key  variable  from  which  three  sub¬ 
categories  were  obtained.  The  APC  code  is  a  three-digit  code  that  summarizes  pertinent 
portions  of  an  Officer’s  prior  college  performance.  The  first  digit  indicates  overall 
academic  performance.  The  second  represents  mathematical  background  and  the  third 
represent  course  coverage  in  science  and  technical  fields.  The  three  separate  digits  reflect 
an  individual’s  cumulative  grade  point  average,  exposure  to  and  performance  in  calculus- 
related  mathematics  and  selected  science/engineering  areas.  Use  of  the  APC  is  limited  in 
that  only  about  40%  of  the  pilots  in  the  data  set  have  been  assigned  this  code.  The 
individual  digits  of  the  APC  code  were  re-classified  into  a  binary  variable  based  upon 
individual  performance.  A  binary  code  of  1  was  assigned  to  individuals  who  demonstrated 
B+  or  better  GPA,  math  and/or  technical  backgrounds. 

Other  variables  were  derived  from  the  data  set  for  ease  in  manipulation.  “RACE” 
and  “ETHNIC”  categorical  variables  were  coded  according  to  major  divisions.  Initially, 
they  consisted  of  numerous  factors,  some  of  which  contained  only  a  few  individuals. 
“RACE”  was  reduced  to  factors  of  “White,”  “Black,”  and  “Other”.  “ETHNIC”  was  coded 
to  allow  “None,”  “Hisp,”  “Asian”  and  “Other”  as  possible  factor  levels.  Commissioning 
Source  was  also  used,  with  factor  levels  of  “OCS,”  “NROTC,”  and  “USNA”  being  created 
for  the  categorical  variable  “SOURCE.”  Finally,  both  “FAR”  and  “AQT”  integer  variables 
were  used  from  results  of  the  ASTB  scores. 

C.  KEY  DEPENDENT  VARIABLE  DESCRIPTION 

Aviator’s  flight  school  completion  status  was  determined  by  “CURRSTAT,”  which 
identified  whether  or  not  an  individual  completed  or  withdrew  from  flight  school.  This 
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categorical  variable  allows  model  formulation  using  either  classification  trees  or  by  Logit 
regression  due  to  its  binary  nature. 

Another  measure  of  effectiveness  was  the  composite  score  obtained  from 
attendance  at  the  initial  flight  training  school  attended.  This  score,  from  the  Navy  Aviation 
Schools  Command  (NASC),  is  based  upon  a  student’s  performance  in  API  and  measures 
academic  performance  in  the  early  stages  of  the  aviation-training  pipeline.  This 
explanatory  variable  was  called  “NASCNSS”  and  is  continuous  in  nature. 

Two  other  continuous  variables  were  used.  The  first,  “PASS,”  reflected  the 
standardized  academic  score  achieved  during  Primary  flight  training,  while  “PFSS”  was  a 
measure  of  flight  performance  during  the  same  period. 

D.  METHODOLOGY 

Statistical  analysis  starts  by  using  Classification  and  Regression  Trees  (CART)  to 
model  performance  of  individuals  in  the  aviator-training  program.  Trees  have  the 
advantage  that  they  are  easier  to  interpret  when  the  predictors  are  a  mixture  of  categorical 
levels  and  numeric  entries.  Furthermore,  the  response  can  either  be  a  continuous  or  a 
categorical  value.  Thus  CART  can  be  used  for  either  “CURRSTAT,”  which  is  a 
categorical  response,  or  for  the  other  response  variables,  which  are  continuous. 

A  tree  is  formed  by  measuring  the  amount  of  variation  or  deviance  between  each 
predictor  and  the  response  variable.  The  predictor  that  reduces  variation  or  deviance  by  the 
greatest  amount  becomes  a  “parent”  node  and  a  partition  is  made,  dividing  the  parent  node 
into  two  “child”  nodes  that  contain  all  the  other  predictor  variables.  Each  child  node  then 
becomes  a  parent  and  the  process  repeats  itself.  This  process  continues  until  there  are  no 
members  of  a  factor  left  to  split  or  if  a  preset  minimum  deviance  reduction  level  is  reached. 
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The  process  is  binary  because  parent  nodes  are  always  split  into  exactly  two  child  nodes 
and  recursive  because  the  process  can  be  repeated  by  treating  each  child  node  as  a  parent. 
(Breiman,  et.  al.,  1 984). 

After  the  initial  tree  model  is  determined,  cross-validation  (CV)  is  used  to  optimize 
the  predictive  reliability  of  the  tree  model.  Several  methods  of  cross-validation  exist; 
however  this  thesis  utilizes  F-fold  cross-v  alidation  (StatSoft,  2000).  This  type  of  cross- 
validation  is  useful  when  no  test  sample  is  available  and  the  learning  sample  is  too  small  to 
have  the  test  sample  taken  from  it.  A  specified  F  value  for  F-fold  cross-validation 
determines  the  number  of  random  sub-samples,  as  equal  in  size  as  possible,  that  are  formed 
from  the  learning  sample.  The  classification  tree  is  computed  V  times,  each  time  leaving 
out  one  of  the  sub-samples  from  the  computations,  and  using  that  sub-sample  as  a  test  set, 
so  that  each  sub-sample  is  used  F  -1  times  in  the  learning  sample  and  just  once  as  the  test 
sample. 

Finally,  the  total  amount  of  deviance  reduction  is  measured  as  each  node  of  the 
crass-validated  tree  is  generated.  As  the  tree  increases  in  size  and  complexity,  an  optimal 
point  will  be  reached.  At  this  point,  the  most  important  splits  in  the  tree  have  been 
determined.  The  tree  is  then  pruned  so  that  only  these  factors  are  used  in  final  tree 
construction.  The  resultant  tree  provides  the  user  a  tool  for  making  predictions  about 
individuals  contained  in  the  data  set.  Further,  the  tree  has  effectively  screened  and 
identified  the  most  useful  predictor  variables  from  the  data.  These  variables  provide  the 
initial  inputs  to  the  Logistic  and  Least-Squares  regression  models  subsequently  formulated. 
Since  the  focus  of  this  study  is  to  determine  the  impact  on  academic  major  and  quality  of 
college  attended,  “COLLEGE”  and  “MAJOR”  are  added  to  the  Logistic  and  Least-Squares 
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models  along  with  the  predictors  identified  by  CART,  if  not  already  identified  by  the  tree 
created. 

The  Logistic  model  uses  the  predictor  variables  identified  by  the  CART  model, 
along  with  the  academic  background  variables  to  get  an  estimated  “Logit”  for  individuals 

in  the  data  set.  This  value  is  analogous  to  the  estimator  Y  that  is  calculated  using  Least- 
Squares  regression.  A  full  model  is  developed,  and  then  a  step  function  is  applied  to 
reduce  this  model,  keeping  only  those  predictors  that  provide  the  most  information  in  the 
model.  This  step  function  uses  the  Akaike  Information  Criterion  (AIC),  and  measures  the 
model  effectiveness,  based  upon  sample  size  and  the  number  of  predictors  in  the  model.  If 
a  predictor  does  not  provide  statistically  significant  information  based  on  the  AIC,  it  is  not 
included  in  the  final  model  formulation  (Agresti,  1990).  The  final  Logistic  model  can  be 
formulated  as: 

A  -  A +P\xn  +Pixa +-+Pk-\xi,k~\ 

Using  an  inverse  transformation,  a  predicted  probability  of  flight  school  success  for 
the  individual  can  be  calculated.  Thus, 


Using  this  transformation,  the  predicted  success  probability  for  each  member  of  the  data  set 
can  be  computed,  based  upon  his  or  her  educational  and  demographic  background  and 
flight  school  performance  data  (Hamilton,  1992). 
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IV.  RESULTS 


A.  DATA  ANALYSIS 

The  student  aviator  data  set  contains  2612  members,  of  which  150  are  women.  Of 
these  members,  2144  completed  training,  while  468  failed  to  complete  the  pipeline.  The 
data  set  examined  did  not  provide  an  explanation  for  early  departure  from  the  flight- 
training  program.  There  were  1235  NROTC  graduates,  991  USNA  graduates,  and  386 
OCS  graduates  in  the  program.  A  complete  breakdown  of  these  and  all  other  elements  of 
the  student  aviator  data  set  can  be  found  in  Appendix  B. 

A  two-way  contingency  table  was  arranged  and  a  chi-square  test  of  independence 
was  conducted,  with  a  null  hypothesis  was  that  there  were  no  differences  between  majors 
based  upon  the  proportions  that  completed  the  flight-training  program.  The  null  hypothesis 
was  rejected,  indicating  that  there  is  a  difference  between  majors  (chi-square  =  29.9662,  df 
=  2,  p-value  =  0).  Comparisons  were  made  to  determine  how  the  majors  differed  from 
each  other.  No  significant  differences  between  non-technical  and  technical  majors  were 
found  (chi-square  =  1.0226,  df  =  1,  p-value  =  0.3119).  However,  between  non-technical 
majors  and  engineers,  significant  differences  exist  (chi-square  =  29.3487,  df  =  1,  p-value  = 
0).  Technical  majors  and  engineers  were  compared,  with  significant  differences  found 
(chi-square  =  6.3579,  df  =  1,  p-value  =  0.0117).  Finally,  technical  majors  and  engineers 
were  combined,  and  then  compared  to  non-technical  majors,  resulting  in  significant 
differences  being  detected  (chi-square  =  23.8068,  df  =  1,  p-value  =  0).  In  summary,  non¬ 
technical  majors  have  significantly  lower  completion  rates  than  the  combined  set  of 
technical  and  engineering  counterparts  (Table  1). 
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Non- 

Technical 

Majors 

Technical 

Majors 

Engineering 

Majors 

Technical 

and 

Engineers 

Total 

1036 

247 

861 

1108 

2144 

Attrited 

285 

57 

126 

183 

468 

Total 

1321 

304 

987 

1291 

2612 

Proportion 

Completing 

Training 

78.43% 

81.25% 

87.23% 

85.82% 

82.08% 

Table  1 .  Student  aviator  completion  rate,  based  upon  Undergraduate  Major.  There  is  no 
evidence  that  differences  exist  between  non-technical  and  technical  majors,  but  do  exist 
with  engineering  majors,  and  between  non-technical  majors  when  compared  against  both 
technical  majors  and  engineers  combined. 


A  CART  was  formed  using  the  “CURRSTAT”  dependent  variable,  indicating 
completion/attrition  status.  The  model  is  as  follows: 

CURRSTAT  ~  SEX  +  AQT  +  FAR  +  RACE  +  MAJOR  +  ETHNIC  +  SOURCE  + 
COLLEGE  +  GPA  +  MATH  +  PHYSICS 

A  full,  saturated  tree  was  generated  and  cross-validated  using  V-fold  CV  with  ten 
holdout  sets.  Figure  1  displays  the  deviance  plot  of  the  cross-validated  tree. 


65.000  5.500  3.600  3.000  2.300  1.500  0,930  0.180 


Figure  1.  Size  vs.  deviance  plot  for 
CURRSTAT  model. 
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The  minimal  point  of  the  deviance  plot  is  five,  indicating  that  a  tree  with  five  terminal 
nodes  provides  the  most  accurate  prediction.  The  tree  was  pruned  to  reflect  this  optimal 
level.  All  variables  were  made  available  to  the  algorithm;  Figure  2  shows  that  only  FAR, 
AQT  and  RACE  were  actually  chosen.  (For  more  details  on  the  construction  of  trees  see 
Breiman  et  al.,  1984.)  Results  indicate  that  the  best  predictors  consist  of  FAR,  AQT  and 
RACE  (Figure  2).  In  this  and  other  tree  pictures,  ovals  represent  non-terminal  nodes  and 
rectangles,  terminal  ones.  Each  node  is  labeled  with  the  proportion  or  average  in  that  node 
-  in  this  case,  the  proportion  of  aviators  undergoing  attrition.  Beneath  each  node  the 
number  of  aviators  falling  into  that  node  is  given. 


Figure  2.  Reduced  Aviator  CURRSTAT  Tree,  with  FAR,  AQT 
and  RACE  as  significant  predictors  of  completion  and  attrition. 
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The  above  figure  indicates  that  those  individuals  who  score  six  or  less  on  the  FAR  have  a 
24.4%  attrition  rate,  while  those  who  score  7  or  above  have  a  12.3%  attrition  rate.  Further, 
individuals  who  have  a  score  or  4  or  less,  given  that  they  have  scored  a  6  or  less  on  the 
FAR  have  a  38.6%  attrition  rate.  Finally,  those  individuals  who  are  “Black”  or  “Other”  in 
the  RACE  category  are  predicted  to  have  a  54.8%  attrition  rate,  given  that  they  have  a 
score  of  6  or  less  on  the  FAR  and  a  4  or  less  on  the  AQT.  The  tree  model  indicates  that  the 
Navy  should  focus  efforts  to  recruit  individuals  with  higher  FAR  and  AQT  scores. 

After  CART  optimal  tree  determination,  a  Logit  model  was  formed  using  FAR, 
AQT,  and  RACE,  along  with  MAJOR  and  COLLEGE.  A  full  model  was  created  using 
single  predictors  and  their  two-level  interactions.  Following  creation  of  this  model,  a  step 
function  using  the  AIC  was  utilized  to  determine  the  optimal  reduced  model.  Final  model 
formulation  is: 

L,  =  po  +  ptFAR  +  P2AQT  +  faRACEOther  +  p.RACEWhite  +  psMAJORl  +  p. MAJOR  2 
+  p COLLEGE  1  +  p%COLLEGE  2  +  BgFAR  :  AQT  +  PX0FAR  :  COLLEGE  1  +  puFAR  :  COLLEGE  2 

Results  from  the  Logistic  regression  model  indicate  that  engineering  majors  perform 

significantly  better  than  individuals  who  hold  non-technical  degrees  (MAJOR2  (t(2600)= 

3.39,  p  <  0.001).  Individuals  who  attended  the  Naval  Academy  also  performed  better  than 

those  who  attended  non-ranked  civilian  institutions  (COLLEGE2  (t(2600)  =  2.75,  p  < 

0.01).  In  addition,  FAR  (t(2600)  =  5.51,  p  <  0.001),  AQT  (t(2600)  =  2.48,  p  <  0.01)  and 

RACEWhite  (t(2600)  =  4.03,  p  <  0.001)  are  significant.  In  this  model,  “White”  individuals 

perform  significantly  better  than  their  “Black”  counterparts.  In  addition  to  the  single 

variable  predictors  above,  a  significant  negative  interaction  was  observed  between  FAR 

and  AQT  (t(2600)  =  -3.14,  p  <  0.01).  Finally,  an  interesting  interaction  was  observed 
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between  individuals  who  attended  the  Naval  Academy  and  how  they  performed  on  the 
FAR.  Results  indicate  that  each  point  increase  on  the  FAR  decreases  the  log-odds  by  0.165 
(t(2600)  =  -2.25,  p  <  0.05).  Reasons  for  this  are  unclear.  Table  2  gives  a  listing  of  the 
above  independent  variables,  their  coefficient  estimates  B, ,  their  resulting  t-statistics  and 
finally,  their  level  of  significance. 


LOGIT 

Std.  Error 

t  value 

Significance 

(Intercept) 

-3.8352 

0.8478 

-4.52 

NS 

FAR 

0.7527 

0.1366 

5.51 

*** 

AQT 

0.3862 

0.1557 

2.48 

RACEOther 

0.1256 

0.2807 

0.48 

NS 

RACEWhite 

0.8482 

0.2107 

4.03 

*** 

MAJOR1 

0.1259 

0.1696 

0.74 

NS 

MAJOR2 

0.4223 

0.1246 

3.39 

*** 

COLLEGE1 

-0.5809 

0.6003 

-0.97 

NS 

COLLEGE2 

1.2965 

0.4712 

2.75 

** 

FAR:AQT 

-0.0710 

0.0226 

-3.14 

*** 

FARCOLLEGE1 

0.0753 

0.0888 

0.848 

NS 

FARCOLLEGE2 

0.1651 

0.0735 

-2.25 

* 

***  Significant  at  the  alpha  =  0.001  level 
**  Significant  at  the  alpha  =  0.01  level 
*  Significant  at  the  alpha  =  0.05  level 
NS  Not  Significant 

Table  2.  Logit  regression  results  for  Aviator  CURRSTAT 
model. 


The  results  of  the  Logit  model  can  be  used  by  the  Naval  Recruiting  Command  to  set 
quotas  for  student  naval  aviator  billets.  Each  year,  the  Chief  of  Naval  Operations  Aviation 
Air  Warfare  identifies  the  number  of  student  aviator  billets  required  to  maintain  aviation 
military  readiness  in  the  future.  There  are  fewer  billets  available  due  to  the  military  budget 
reduction,  thus  manpower  analysts  must  maintain  high  retention  rates  for  each  aviation 
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year  group  or  there  will  be  aviator  shortages  several  years  later.  An  aviation  manpower 
analyst  can  use  this  Logit  model  to  predict  whether  or  not  an  individual  will  successfully 
complete  flight  school  training.  By  selecting  individuals  who  have  a  higher  probability  of 
completing  flight  school,  the  Navy  may  not  experience  the  current  retention  problems  that 
has  plagued  the  aviation  community  these  past  few  years.  For  example,  suppose  a  recent 
white  USNA  graduate  with  an  engineering  degree  scored  a  six  on  both  the  FAR  and  on  the 
AQT.  Using  these  characteristics,  a  resulting  Logit  for  this  individual  can  be  calculated 
using  the  model  formulation  below: 

L,  =  0O  +  p]FAR  +  P2AQT+ &RACEOther+  pARACEWhite+  &MAJOR  +  &MAJOK2 
+  fijCOLLEGEl  +  p, COLLEGE!  +  p9FAR :  AQT+ P]0FAR :  COLLEGE 1  +  pnFAR :  COLLEGEl 

L  =  -3.8352  +  0.7527*  6  +  0.3862  *  6  +  0. 1256  *  0  +  0.8482  *  1  +  0. 1 259*  0  +  0.4223  *  1 
+  (-0.5809)  *0  +  1 .2965  *  1  +  (-0.07 1 0)  *  6  *  6  +  0.07530  *  0  +  (-0. 1 65 1)  *  6 
=  2.0186 


Performing  the  inverse  transformation: 


1 


1 


-2.0186 


=  0.883 


\  +  e~L  l  +  e~ 

From  this,  it  is  predicted  that  this  individual  would  have  a  success  probability  of  88%. 

On  the  other  hand,  if  a  recent  black  student  who  graduated  from  a  non-ranked  institution 
with  a  non-technical  degree  had  scored  a  4  on  the  AQT  and  a  5  on  the  FAR  was 
considered.  Calculating  the  appropriate  Logit: 
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L,  =0O+  0FAR  +  p2AQT+ fi,RACEOther+  j3ARACEWhite+  &MAJORI  +  J3bMAJOR2 
+  ^COLLEGE  +  &COLLEGEI  +  &FAR  :AOT+ ft0FAR :  COLLEGE  +  ft  tFAR :  COLLEGE 2 

L  =  —3.8352  +  0.7527  *  5  +  0.3862  *  4  +  0. 1 256  *  0  +  0.8482  *  0  +  0. 1 259  *  0  +  0.4223  *  0 
+  (-0.5  809)  *0  +  1 .2965  *  0  +  (-0.07 1 0)  *  5  *  4  +  0.07530  *  0  +  (-0. 1 65 1)  *  0 
=  0.5310 


Performing  the  inverse  transformation: 


P  = 


1  +  e 


-L 


1  +  e 


0.5310 


=  0.6297 


This  individual’s  predicted  success  is  63%. 

The  next  analysis  of  the  student  aviator  data  dealt  with  the  results  of  the  pilots  who 
ultimately  completed  training  on  their  first  composite  grade  from  aviation  flight  school. 
Model  formulation  was  the  same  as  above,  but  this  time  using  the  standardized  score, 
NASCSSC,  as  the  dependent  variable.  Model  formulation  is: 

NASCSSC  ~  SEX  +  AQT  +  FAR  +  RACE  +  MAJOR  +  ETHNIC  +  SOURCE  + 
COLLEGE  +  GPA  +  MATH  +  PHYSICS 

This  score  is  the  first  that  student  aviators  receive  upon  admittance  into  the  flight-training 
program.  It  represents  a  time  when  they  are  most  competitive,  and  are  striving  to  perform 
at  their  peak  in  order  to  have  the  best  pipeline  selection  choice.  Results  for  deviance 
reduction  are  similar  to  the  findings  above,  with  an  optimal  tree  of  three  terminal  nodes 
created.  Most  important  are  FAR  and  AQT.  This  result  is  not  surprising,  since  the  purpose 
of  the  AQT  and  the  FAR  is  to  predict  performance  in  flight  school  during  the  API  and 
primary  phases.  Individuals  who  score  seven  or  less  on  the  FAR  are  predicted  to  have  a 
NASCNSS  score  of  51.49%,  while  individuals  who  score  above  seven  are  predicted  to 
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have  a  score  of  54.53%.  Further,  individuals  who  score  five  or  less  on  the  AQT,  given  that 
they  have  scored  a  seven  or  less  on  the  FAR  are  predicted  to  have  a  score  of  49.88%.  The 
trend  indicates  that  higher  FAR  and  AQT  scores  lead  to  higher  composite  scores  achieved 
during  API.  Figure  3  shows  the  reduced  tree  model. 


Figure  3.  Reduced  NASCNSS  tree  model. 

Indicates  that  only  FAR  and  AQT  are  significant 
predictors  of  API  performance. 

A  plot  of  the  normalized  distribution  of  the  residuals  was  made  for  the  NASCNSS 
data.  Results  (Figure  4)  indicate  highly  skewed  tails  on  both  ends  of  the  data  set.  A  review 
of  the  original  data  indicated  that  some  individuals  had  very  low  scores,  yet  had  completed 
the  training  pipeline.  Similarly,  some  individuals  had  high  composite  scores,  while  their 
raw  score  was  lower.  This  may  be  indicative  of  poor  data  collection  and  entry  efforts  and 
adversely  impacts  the  significance  of  this  finding. 
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-2  0  2 
Quantiles  of  Standard  Normal 

Figure  4.  Standard  normal  plot  of  quantiles  vs. 

,  residuals  for  the  NASCNSS  Model. 

Least-Squares  regression  on  the  NASCNSS  model  was  conducted.  A  step  function 
was  utilized  to  determine  the  optimal  linear  model,  using  the  AIC.  The  final  model 
included  FAR,  AQT,  MAJOR  and  COLLEGE,  with  a  significant  interaction  between  FAR 
and  AQT.  Both  FAR  (t(2136)  =  4.87)  and  AQT  (t(2136)  =  4.66)  are  significant  with 
resultant  p- values  =  0.00.  Technical  degreed  aviators  (t(2136)  =  2.08,  p  <  0.05)  and 
engineers  (t(2 136)  =  2.53,  p  <  0.05)  were  also  significant.  Of  note  is  that  there  is  no 
significant  effect  from  individuals  that  attended  the  USNA  as  compared  to  the  baseline 
group.  However,  individuals  who  attended  top  ranked  institutions  performed  significantly 
better  than  those  who  attended  non-ranked  universities  (t(2 136)  =  3.04,  p  <  0.005).  Finally, 
a  high  degree  of  correlation  was  evident  between  FAR  and  AQT  (0.9143),  leading  to  a 
significant  interaction  between  these  two  terms  (t(2136)  =  -2.93,  p  <  0.005).  A  possible 
explanation  for  this  result  is  that  some  individuals  might  score  high  on  one  predictor  but 
low  on  the  other;  however,  this  is  not  entirely  clear  from  the  analysis.  The  total  variation 
explained  by  the  model  was  only  12%,  indicating  that  the  model  does  not  fit  very  well.  A 
summary  of  the  data  output  is  displayed  in  table  3. 


29 


Value 

Std.  Error 

t  value 

Pr(>|t|) 

(Intercept) 

35.750 

2.574 

13.887 

0.000 

FAR 

1.740 

0.357 

4.870 

0.000 

AQT 

2.075 

0.445 

4.662 

0.000 

MAJOR1 

0.458 

0.221 

2.076 

0.038 

MAJOR2 

0.261 

0.103 

2.534 

0.011 

COLLEGE1 

0.546 

0.180 

3.036 

0.002 

COLLEGE2 

0.693 

0.097 

0.715 

0.475 

FAR:AQT 

-0.175 

0.060 

-2.932 

0.003 

Table  3.  Least-Squares  regression  results  for 
aviator  NASCNSS  model. 


All  pilots,  after  completion  of  API,  transition  to  primary  flight  training.  In  primary, 
the  real  education  of  a  pilot  begins.  Students  are  introduced  to  the  flight  trainer  and 
actually  spend  time  in  the  cockpit  learning  to  develop  flying  skills.  In  addition,  academic 
work  continues,  and  students  receive  a  composite  score  for  each  part  during  this  phase  of 
the  training  program.  The  ASTB  is  designed  to  use  AQT  to  predict  academic  performance, 
while  the  FAR  is  supposed  to  predict  performance  in  the  flight  school  portion  of  primary. 
Of  interest  is  whether  or  not  these  characteristics  hold  true,  and  see  if  there  are  other 
significant  predictors  using  CART.  The  academic  model  formulation  is: 

PASS  ~  SEX  +  AQT  +  FAR  +  RACE  +  MAJOR  +  ETHNIC  +  SOURCE  +  COLLEGE 
+  GPA  +  MATH  +  PHYSICS 

Analysis  followed  the  same  methodology  as  above,  and  an  optimal  minimal  spanning  tree 

with  nine  terminal  nodes  was  developed.  Results  indicate  that  individuals  who  score  six  or 

lower  and  are  not  “White”  score  the  lowest,  with  a  predicted  score  of  43.55%.  On  the 

other  hand,  “White”  individuals  who  score  below  a  seven  on  the  FAR  have  scores  ranging 

from  48.28%  to  57.03%  based  on  their  AQT  score.  Lower  AQT  scores  are  indicative  of 

lower  scores  during  the  academic  portion  of  Primary.  For  individuals  that  score  a  seven  or 

above  on  the  FAR  and  have  a  technical  or  an  engineering  degree,  they  are  predicted  to 
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score  a  55.42%  during  this  training  phase.  Further,  if  they  score  eight  or  higher  on  the 
AQT,  they  are  predicted  to  finish  with  a  57.26%  final  score.  Close  examination  of  the  tree 
model  indicates  that  lower  FAR  and  AQT  scores  result  in  lower  predicted  composite  scores 
achieved  during  the  academic  portion  of  primary  (Figure  5).  Finally,  using  the  normalized 
QQ  plot  to  inspect  the  distribution  of  residuals,  it  is  apparent  that  there  is  one  point  that 
requires  further  investigation  (Figure  6).  A  review  of  the  data  set  indicated  that  this 
individual  had  the  highest  standardized  score,  while  his  raw  score  was  lower  than  that  of 
many  of  his  peers.  It  is  believed  that  this  is  another  case  where  faulty  data  entry  is  causing 
the  discrepancy. 


Figure  5.  Reduced  PASS  tree  model.  Results  indicate  that  FAR, 
RACE,  MAJOR,  and  AQT  are  significant  factors  in  tree 
construction. 
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Figure  6.  Standard  normal  plot  of  quantiles  vs. 
residuals  for  the  PASS  model. 


A  Least  Squares  regression  model  was  again  formulated  using  the  techniques 
described  above  with  the  results  shown  in  table  4  below.  Independent  variables  used  in 
model  selection  include  FAR,  MAJOR,  AQT,  RACE  and  COLLEGE.  Notice  that  all 
factors  are  significant  at  the  alpha  =  0.05  level  or  better  in  this  model.  However,  the  total 
variation  explained  by  the  model  was  only  8%,  indicating  that  while  statistically 


significant,  there  is  question  about  the  usefulness  of  this  result. 


Value  Std.  Error  t  value  Pr(>|t|) 


(Intercept) 

30.8379 

1.2957 

23.7997 

0 

FAR 

0.9016 

0.1254 

7.1904 

0 

MAJOR1 

1.8064 

0.6129 

2.9474 

0.0032 

MAJOR2 

2.6708 

0.4101 

6.513 

0 

RACEOther 

2.7437 

1.3982 

1.9624 

0.0498 

RACEWhite 

6.2315 

1.056 

5.901 

0 

AQT 

1.1012 

0.1557 

7.0736 

0 

COLLEGE1 

1.4543 

0.4989 

2.9149 

0.0036 

COLLEGE2 

1.4562 

0.4397 

3.3115 

0.0009 

Table  4.  Least-Squares  regression  results  for  aviator 
PASS  model. 
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The  final  analysis  was  conducted,  similar  to  the  methods  above,  on  the  PFSS 
dependent  variable.  FAR  is  expected  to  be  a  significant  predictor  of  these  scores,  based 
upon  the  validity  of  the  ASTB.  Model  formulation  is: 

PFSS  ~  SEX  +  AQT  +  FAR  +  RACE  +  MAJOR  +  ETHNIC  +  SOURCE  +  COLLEGE 
+  GPA  +  MATH  +  PHYSICS 

Again,  there  is  significant  deviance  reduction,  with  an  optimal  tree  of  nine  nodes.  A  tree 
grown  using  this  value  resulted  in  FAR  ,  RACE,  COLLEGE,  MAJOR,  and  ETHNIC  being 
the  important  factors  in  this  model.  Individuals  who  score  six  or  less  on  the  FAR  and  are 
not  “White”  are  predicted  to  score  lowest  among  all  groups.  Non-technical  majors  who 
attend  non-ranked  colleges  also  fare  poorly,  if  they  score  a  seven  on  the  FAR,  as  compared 
to  their  technical  and  engineering  counterparts  who  fall  within  the  same  category.  The 
minimal  spanning  tree  plot  is  displayed  below  (Figure  7)  along  with  the  normalized  QQ 
plot  (Figure  8). 

A  Least  Squares  regression  analysis  was  conducted  on  the  PFSS  data  using 
predictors  FAR,  AQT,  MAJOR,  ETHNIC  and  COLLEGE.  Use  of  the  step  model  removed 
AQT  from  consideration.  Results  indicate  that  FAR  was  highly  significant  (t(2129)  = 
15.62,  p  =  0.00)  while  MAJOR  did  not  have  an  effect  on  performance  during  PFSS.  There 
was  a  significant  effect  of  the  RACE  predictor,  with  “White”  (t(2129)  =  4.44,  p  0  0.00)  and 
“Other”  (t(2129)  =  2.55,  p  <  0.01)  categories  performing  significantly  better  than  “Blacks.” 
In  addition,  the  USNA  graduates  fare  better  than  their  non-ranked  school  counterparts  at 
the  alpha  =  0.10  level.  Significant  interactions  were  present.  Individuals  who  attended  the 
USNA  that  held  technical  or  engineering  degrees  performed  better  than  their  non-technical 
counterparts.  Finally,  it  is  important  to  note  that  these  predictors  explained  only  16%  of 
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Figure  7.  Reduced  PFSS  tree  model.  Results  indicate  that  FAR, 
RACE,  MAJOR,  and  AQT  are  significant  factors  in  tree 
construction.' 
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Quantiles  of  Standard  Normal 

Figure  8.  Standard  normal  plot  of  quantiles  vs. 
residual  for  PFSS  model.  Some  outliers  are 
present,  as  indicated  by  the  skewed  plot  tails. 


the  total  variation  in  the  data  set  as  evidenced  by  the  value  of  R2  that  was  calculated  for  this 
regression  model.  Table  5  displays  the  output  for  this  model. 


Value  Std.  Error  t  value  Pr(  >  |t| ) 


(Intercept) 

28.6916 

2.2066 

13.0025 

0 

FAR 

1.8821 

0.1205 

15.6162 

0 

MAJOR  1 

-0.5652 

1.0826 

-0.5221 

0.6017 

MAJOR2 

0.7012 

0.6531 

1.0737 

0.2831 

RACEOther 

4.3549 

1.7067 

2.5517 

0.0108 

RACEWhite 

4.7542 

1.0705 

4.441 

0 

COLLEGE1 

0.8577 

0.6979 

1.2289 

0.2192 

COLLEGE2 

1.2226 

0.6749 

1.8114 

0.0702 

ETHNICHisp 

-0.1234 

1.8008 

-0.0685 

0.9454 

ETHNICNone 

3.9332 

1.8427 

2.1345 

0.0329 

ETHNICOther 

2.6099 

1.8536 

1.408 

0.1593 

MAJOR1COLLEGE1 

2.3856 

1.9485 

1.2243 

0.221 

MAJOR2COLLEGE1 

-1.0833 

1.0511 

-1.0306 

0.3028 

MAJOR1COLLEGE2 

2.4013 

1.3951 

1.7212 

0.0854 

MAJOR2COLLEGE2 

1.6045 

0.9298 

1.7256 

0.0846 

Table  5.  Least-Squares  regression  results  for  aviator  PFSS 
model. 
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V. 


DISCUSSION 


A.  SUMMARY 

The  present  policy  of  the  Aviator  Recruiting  Command  is  to  recruit  those 
individuals  most  likely  to  succeed  in  the  flight-training  program,  based  upon  their  ASTB 
scores  and  physical  attributes.  In  addition,  emphasis  is  placed  on  recruiting  individuals 
having  “technical”  undergraduate  majors.  This  follows  the  “Rickover  Hypothesis,”  that 
individuals  with  technical  degrees  make  better  officers  than  their  non-technically  educated 
counterparts.  The  recruiting  focus  is  supposed  to  be  placed  on  individuals  with  technical 
degrees;  however,  it  is  apparent  that  this  precept  is  not  followed  because  of  the  nearly  equal 
proportions  of  aviators  with  non-technical  majors  and  those  with  technical  or  engineering 
majors  combined.  This  distribution  provides  the  ability  to  make  a  direct  comparison 
between  the  performances  of  each  group.  The  evidence  indicates  that  aviators  with 
engineering  degrees  have  a  greater  propensity  for  completing  the  training  pipeline  than 
their  technical  and  non-technical  counterparts.  This  echoes  the  conclusion  found  by 
Woelper  (1998),  who  determined  that  SO’s  with  engineering  degrees  had  higher 
completion  rates  than  their  non-engineering  counterparts  in  the  nuclear  power  training 
process.  However,  he  also  indicated  that  no  effect  on  performance  was  observed  after 
completion  of  training,  indicating  that  the  effect  of  undergraduate  major  diminishes  as  one 
progresses  upward  in  his  or  her  Naval  Career  (Woelper,  1998). 

Many  factors  affect  pilot  performance  during  the  training  process;  however,  they 
are  numerous  and  not  well  defined.  Initial  review  of  the  above  results  seems  to  indicate 
that  MAJOR  and  COLLEGE  are  significant  factors  in  pilot  development;  however,  this 
conclusion  must  be  viewed  with  caution.  Although  the  explanatory  variables  for  each 
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performance  measure  were  statistically  significant,  one  must  ask  whether  or  not  these 
variables  account  for  all  the  variation  observed  by  the  model  formulation.  Because  residual 
deviance  was  very  high  in  the  CART  models,  and  the  corresponding  R2  value  was 
extremely  low  in  all  of  the  least  squares  regression  models,  inferences  about  this  result  are 
difficult  to  make.  However,  certain  observations  can  be  made. 

The  first  observation  to  consider  is  MAJOR.  The  data  suggest  that  individuals  with 
engineering  degrees  perform  better  during  API  and  in  the  academic  portion  of  Primary 
flight  training,  as  indicated  by  significant  findings  in  these  areas.  On  the  other  hand,  the 
effect  of  holding  a  “technical”  degree  was  not  significant  during  the  flight  portion  of 
Primary.  This  indicates  that  while  a  strong  educational  background  in  a  technical  field 
increases  the  likelihood  of  having  higher  academic  grades  and  less  likelihood  of 
withdrawal  from  the  flight-training  program,  there  is  little  relationship  between  this  degree 
and  flight  school  performance  in  the  air. 

The  second  observation  deals  with  the  importance  of  COLLEGE.  In  all  but  the 
NASCNSS  model,  USNA  attendance  was  an  important  predictor  of  success,  whether 
dealing  with  completion  rates,  or  when  comparing  grades.  It  can  be  hypothesized  that 
individuals  who  attended  the  Naval  Academy  have  better  preparation  than  their  NROTC 
and  OCS  counterparts  due  to  the  unique  experience  they  receive  through  attendance  and 
participation  in  the  military  institution.  In  short,  they  perform  better  because  they  are  more 
disciplined  and  require  less  time  to  acclimate  to  the  military  surroundings  than  their 
counterparts,  resulting  in  greater  attention  to  the  primary  task  at  hand.  It  is  important  to 
note  that  SOURCE  was  not  a  factor  named  in  any  of  the  CART  models,  indicating  that 
there  is  not  much  difference  when  approached  from  this  angle.  However,  evidence 
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suggests  that  individuals  who  attend  top-ranked  institutions  and  those  who  attend  the 
USNA  fare  better  than  their  counterparts,  regardless  of  whether  or  not  they  entered  through 
OCS  orNROTC. 

Finally,  both  the  AQT  and  FAR  measures  were  identified  in  the  CART  models, 
leading  to  the  belief  that  they  are  performing  their  job  as  predictors  of  success  in  the 
aviation  program.  The  results  of  this  study  seem  to  further  validate  these  measures  of 
performance  and  suggest  that  the  ASTB  be  retained  as  an  initial  screening  tool  for  aviators. 

Several  issues  and  problems  with  the  above  conclusions  need  to  be  discussed.  First 
is  the  issue  of  data  accuracy  and  completeness.  In  many  instances,  large  blocks  of  the  data 
set  had  to  be  excluded  due  to  missing  data  or  other  inaccuracies  in  the  data  files.  For 
instance,  795  members  of  the  NAMRL  data  set  were  unable  to  be  matched  by  social 
security  number  in  the  OMF  data  files,  even  though  they  were  listed  as  being  naval 
officers.  Numerous  other  instances  occurred  where  individuals  were  listed  as  completing 
the  flight-training  pipeline,  but  did  not  have  NASCNSS,  PFSS,  or  PASS  grades  entered. 
Another  problem  was  that  some  individuals  were  listed  as  having  attended  the  USNA,  as 
identified  by  COLLEGE,  yet  these  same  individuals  were  listed  as  having  commissioning 
source  of  OCS  or  NROTC.  All  of  these  individuals  were  removed  from  consideration 
during  analysis. 

Another  important  issue  is  the  question  of  sample  size.  In  statistical  analysis,  a 
large  sample  size  enables  one  to  detect  even  small  differences  between  measures  of 
performance,  based  on  explanatory  variables.  However,  judgment  must  be  made  to 
determine  if  statistically  significant  results  are  of  practical  use.  These  results  must  be 
viewed  with  this  in  mind.  A  population  of  2612  pilots  provides  ample  room  to  find  even 
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minute  differences  between  groups  of  people.  Further,  the  population  from  which  the 
results  were  derived  must  be  representative  of  the  set  of  all  aviators  who  attended  flight 
school.  In  conclusion,  it  is  believed  that  differences  do  exist  between  pilots,  based  upon 
their  academic  background.  However,  the  practical  necessity  of  recruiting  pilots  who  meet 
the  minimum  standards  to  maintain  required  manning  levels  requires  focus  on  those  that 
might  not  have  attributes  that  are  strong  predictors  of  success  as  determined  from  this 
analysis. 

A  third  issue  deals  with  the  absence  of  data  reflecting  actual  performance  of  an 
individual  during  his  or  her  undergraduate  education.  The  APC  is  supposed  to  provide 
information  regarding  overall  academic  performance,  in  addition  to  performance  in 
mathematics  and  physics.  Only  40%  of  the  individuals  in  the  data  set  held  the  APC.  The 
APC  is  computed  by  the  Naval  Postgraduate  School  (NPS),  usually  within  three  years  from 
commissioning.  However,  individuals  must  ensure  that  college  transcripts  are  forwarded  to 
the  school  for  computation.  This  score  is  required  for  acceptance  into  NPS  and  for 
consideration  of  acceptance  into  other  graduate  school  programs.  An  individual  who  does 
not  desire  to  pursue  a  graduate  education  might  not  request  that  their  transcripts  be 
forwarded.  It  is  possible  that  a  significant  number  of  individuals  who  lack  this  code  might 
have  lower  grades,  and  therefore  consider  themselves  ineligible  for  graduate  education. 
The  APC  and  its  sub-scores  of  GPA,  MATH  and  PHYSICS  were  not  utilized  in  final 
model  formulation;  however,  with  a  more  complete  data  set,  this  information  could  have 
been  used  to  develop  a  more  accurate  picture  of  flight  school  performance. 

Most  important,  however,  is  the  issue  of  the  determination  of  completion  and 
attrition.  The  data  sets  provided  did  not  provide  reasons  for  attrition,  necessitating  all  those 
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who  withdraw  be  included  in  the  model.  Identification  of  those  who  attrite  for  medical 
reasons,  or  reasons  not  related  to  individual  performance,  would  hopefully  provide  more 
useful  data  for  analysis.  This  would  limit  the  analysis  to  that  of  those  individuals  who  had 
difficulty  in  performing  academic  or  flight-related  tasks  and  enable  a  more  precise 
identification  of  factors  related  to  successful  flight  school  completion.  Nevertheless,  the 
models  do  indicate  a  relationship  between  AQT  and  FAR  and  flight  school  performance, 
indicating  that  these  measures  are  performing  as  designed.  The  factors  of  MAJOR, 
COLLEGE,  and  RACE  also  provide  predictive  power.  Even  without  knowing  the  cause  of 
attrition,  inferences  can  be  made  on  an  individual’s  performance,  based  upon  the  factors 
listed  above.  This  model  does  not  attempt  to  predict  attrition,  however.  It  merely  identifies 
factors  that  indicate  successful  flight  school  completion  and,  using  the  CART 
methodology,  classifies  individuals  into  groups  with  common  characteristics. 

B.  RECOMMENDATIONS 

The  present  recruiting  policy  for  naval  aviators  is  to  place  emphasis  on  recruiting 
individuals  with  technical  degrees.  This  policy  is  sound,  given  the  above  results.  There  are 
limitations  to  this,  as  indicated  by  the  number  of  aviators  with  non-technical  degrees  who 
are  already  in  the  fleet.  It  is  impossible,  in  today’s  difficult  recruiting  environment,  to  find 
only  those  individuals  who  have  engineering  degrees  and  otherwise  meet  all  entrance 
specifications.  The  CART  and  Logit  models  should  be  utilized  to  compare  traits  of 
individuals,  recognizing  that  each  of  the  significant  factors  in  this  model  is  associated  with 
the  individual’s  predicted  success  likelihood.  Those  individuals  who  have  characteristics 
associated  with  higher  success  probability  or  to  higher  predicted  composite  scores  should 
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be  recruited,  if  possible.  Focus  should  be  given  to  those  individuals  who  are  predicted  to 
perform  the  best  in  the  training  program,  as  described  below. 

There  are  several  factors  that  seem  to  provide  consistent  improvements  throughout 
this  analysis.  These  include  AQT,  FAR,  RACE,  MAJOR  and  COLLEGE.  Higher  ASTB 
scores,  which  predict  success  likelihood  in  both  the  academic  and  the  flight  school  portions 
of  the  training  pipeline,  result  in  higher  predicted  completion  rates  and  higher  composite 
scores.  Individuals  who  have  technical  degrees,  or  who  attend  the  Naval  Academy  also 
fare  better  than  their  counterparts.  Neither  of  these  results  is  surprising.  Individuals  who 
are  White  have  higher  completion  rates,  and  higher  composite  scores,  too.  This  result  was 
unanticipated  and  the  reasons  for  these  differences  are  not  clear.  Perhaps  it  reflects  the 
contribution  of  other,  unmeasured  attributes  of  the  aviators;  possibly  it  reflects  cultural 
biases  in  the  tests  themselves.  This  thesis  does  not  recommend  against  selection  of  racial 
minorities;  however  it  does  suggest  that  the  selection  process  can  take  into  account  an 
individual’s  background  and  use  the  Logit  or  CART  model  to  estimate  success  probability. 
Students  with  low  estimated  probabilities  are  natural  targets  for  intervention  strategies 
designed  to  improve  retention  and  thereby  preserve  the  Navy’s  investment. 

There  are  some  areas  that  need  to  be  analyzed  further.  First,  this  study  only  looks  at 
pilots,  and  not  at  Naval  Flight  Officers.  However,  during  the  analysis,  some  statistical 
work  was  done  relating  to  this  group  of  individuals  in  which  results  similar  to  those  found 
above  were  obtained.  A  more  detailed  analysis  should  be  conducted  before  conclusions 
about  the  NFO  community  are  reached. 

Second,  greater  emphasis  needs  to  be  placed  to  ensure  accuracy  and  completeness 
of  data  entry.  All  statistical  analysis  can  only  be  as  good  as  the  data  it  models.  It  is 
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therefore  recommended  that  the  data  collection  process  be  reviewed  and  improvements 
made  in  order  to  improve  the  process  of  statistical  analysis. 

Individual  pilots  maintain  training  folders;  however,  these  folders  are  kept  locally  at 
each  command  and  are  not  compiled  into  a  central  database.  There  is  no  methodology  for 
tracking  aviator  fleet  performance,  such  as  safety  check  flights,  re-qualification  exams, 
upgrades  and  performance  qualifications.  It  is  recommended  that  a  centralized 
methodology  for  tracking  the  performance  and  qualifications  of  pilots  be  developed,  with 
the  ability  to  measure  trends  in  performance  after  the  qualified  pilot  leaves  flight  school. 
This  database  would  provide  a  useful  source  for  measuring  trends  in  pilot  performance 
throughout  the  fleet,  and  give  a  way  of  tracking  trends  occurring  in  the  pilot  community. 
In  addition,  it  would  provide  managers  a  possible  way  to  identify  problem  aviators  before 
they  create  a  crisis. 

Most  important,  however,  is  the  need  to  follow  up  on  the  individuals  included  in 
this  study  for  the  purpose  of  tracking  them  throughout  their  careers.  The  pilots  included  in 
this  study  have  only  begun  their  obligated  service  requirement,  and  have  therefore  not 
reached  major  career  decision  points.  Identification  of  trends  in  retention  and  promotion, 
as  related  to  flight  school  experience,  is  an  area  of  study  that  should  be  examined.  At  issue 
is  whether  or  not  those  individuals  who  performed  well  in  flight  school,  based  upon  their 
composite  scores,  along  with  their  demographic  background,  display  measurable  trends 
that  affect  early  promotion  and  retention  likelihood.  In  short,  can  the  results  of  this  study 
be  extended  to  the  career  of  a  pilot?  A  follow-up  study  to  measure  the  promotion  and 
retention  rates  for  these  individuals  should  be  conducted  as  their  career  progresses. 
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APPENDIX  A.  COMBINED  DATA  FILE  DESCRIPTIONS 


Variable 

Description 

Data  Type/code/notes 

SSN 

Social  Security  Number 

Type:  Character 

CURRSTAT 

Current  Status 

Type:  Factor 

Code:  ATTRITION 

COMPLETION 

SEX 

Sex 

Type:  Factor 

Code:  M  -  Male 

F  -  Female 

COMMDATE 

Commissioning  Date 

Type:  Date 

AQT 

Aviation  Qualification  Test 

Type:  Integer 

Code:  0-9  Test  Score  Results 

FAR 

Flight  Aptitude  Rating 

Type:  Integer 

Code:  0  -  9  Test  Score  Results 

NASCNSS 

NASC  Standard  Score  (%) 

Type:  Double 

FINALCOM 

Final  Composite  Score 

Type:  Double 

RACE 

Race 

Type:  Factor 

Code:  black 
white 
other 

MAJOR 

Undergraduate  Major 

Type:  Factor 

Code:  0  -  Non-Technical 

1  -  Technical 

2  -  Engineer 

Note:  Derived  from  OPNAVINST  1530 
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ETHNIC 

Ethnic  Group 

Type:  Factor 

Code:  Asian 

Hisp 

None 

Other 

SOURCE 

Commissioning  Source 

Type:  Factor 

Code:  OCS 

NROTC 

USNA 

GPA 

Grade  Point  Average 

Type:  Factor 

Code:  0  -  Less  than  3.20  GPA 

1  -  Greater  than  3.20  GPA 

MATH 

Mathematics  GPA 

Type:  Factor 

Code:  0  -  Calculus  Grade  B  or  lower 

1  -  Calculus  Grade  B+  or  better 

PHYSICS 

Physics  GPA 

Type:  Factor 

Code:  0  -  Physics  Grade  B  or  lower 

1  -  Physics  Grade  B+  or  higher 

COLLEGE 

Rating  of  College  Attended 

Type:  Factor 

Code:  0  -  Attended  Non-rated  School 

1  -  Attended  top  ranked  National 

or  Liberal  Arts  College 

2  -  Attended  USNA 

NASCRAW 

NASC  Raw  Score  (%) 

Type:  Double 

PFG 

Primary  Raw  Flight  Grade 

Type:  Double 

PAG 

Primary  Raw  Academic 
Grade  (%) 

Type:  Double 

PFSS 

Primary  Flight  Standard 
Score  (%) 

Type:  Double 
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PASS 

Primary  Academic  Standard 
Score  (%) 

Type:  Double 

PIPELINE 

Aircraft  Type 

Type:  Factor 

Code:  Helo 

Jet 

Prop 
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APPENDIX  B 


SUMMARY  OF  PILOT  DATA 


1.  Data  below  include  all  members  of  the  student  aviator  data  set,  including  both 
attritions  and  completions. 


SSN 

Length:  2612 

Class:  Asls 

Mode : character 


CURRSTAT 

SEX 

COMMDATE 

ATTR:  468 

F:  150 

Min . : 10960 

COMP : 214  4 

M : 24  62 

1st  Qu. : 11840 
Median: 12380 
Mean: 12420 
3rd  Qu . : 13010 
Max. : 14230 

AQT 

FAR 

NASCNSS 

FINALCOM 

Min . : 3 . 

000 

Min 

.  :  3 . 000 

Min . :  0.00 

Min. :  0.0 

1st  Qu . : 5 . 

000  1st 

Qu 

. : 5 . 000  1st 

Qu. : 47 . 00  1st 

Qu. : 162 . 0 

Median : 6 . 

000  Median: 7 . 000  Median:53.00  Median:193.0 

Mean : 6 . 

072 

Mean: 6 . 777 

Mean: 50. 22 

Mean: 166.7 

3rd  Qu . : 7 . 

000  3rd 

Qu 

. :8.000  3rd 

Qu. : 57 . 00  3ro 

Qu. : 2 15 . 4 

Max . : 9 . 

000 

Max 

.  :  9 . 000 

Max. : 97 . 83 

Max. : 304 . 0 

RACE 

MAJOR 

ETHNIC 

SOURCE  GPA  MATH 

Black:  122 

0:1321 

Asian:  59 

NROTC : 1235  : 

1576  :  157  6 

Other:  121 

1:  304 

Hisp:  118 

OCS :  386  1: 

316  1:  220 

White: 2369 

2:  987 

None : 1610 

USNA:  991  0: 

720  0:  816 

Other:  825 

PHYSICS 

COLLEGE 

PIPELINE 

NASCRAW 

PFG 

:  157  6 

0:1042 

NPH : 77  6 

Min. :  0.00 

Min. : 0 . 000 

1:  303 

1:  579 

NPM: 607 

1st  Qu . : 57 . 82 

1st  Qu. : 3 . 034 

0:  733 

2:  991 

NPJ : 382 

Median: 91 . 00 

Median : 3 . 063 

NP0 : 241 

Mean : 77 . 58 

Mean: 2 . 828 

NPA: 204 

3rd  Qu. : 95 . 00 

3rd  Qu. : 3 . 088 

NPT : 14  8 

Max. : 99 . 99 

Max . : 3 . 991 

(Other) : 254 

NA' s : 9 . 000 

PAG 

PFSS 

PASS 

PIPELINE 

Min.  : 

0.00 

Min. :  0.00 

Min . :  0.0 

:  56 

1st  Qu . : 

90.67 

1st  Qu . : 4 1 . 00 

1st  Qu. : 44 . 0 

helo:  776 

Median : 

94 . 00 

Median: 49.00 

Median : 52 . 0 

j  et : 1029 

Mean: 

87.53 

Mean: 45 . 85 

Mean: 48.1 

prop:  751 

3rd  Qu. : 

96.27 

3rd  Qu. : 56. 00 

3rd  Qu. : 59 . 0 

Max .  : 

100.00 

Max. : 99 . 00 

Max. : 95 . 0 

NA's: 

10.00 

NA 1 s : 42 . 00 

NA T  s : 4  2 . 0 
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2.  Data  below  include  all  members  of  the  student  aviator  data  set  who  have  completed 
the  entire  training  pipeline. 


SSN  CURRSTAT  SEX  COMMDATE 


Length : 

2144  ATTR :  0 

F: 

120 

Min. 

:  10960 

Class : 

Asls  COMP: 2144 

M: 

2024 

1st  Qu. 

:  11820 

Mode : 

character 

Median 

.*12200 

Mean 

:  1234  0 

3rd  Qu. 

:  12930 

Max. 

:  13870 

AQT 

FAR 

NASCNSS 

FINALCOM 

Min . 

:  3 . 00 

Min. 

3.000 

Min . 

:  20 . 00 

Min. :  90.0 

1st  Qu. 

:  5 . 00 

1st  Qu. 

6.000 

1st  Qu. 

:  4  9 . 00 

1st  Qu. : 181 . 0 

Median 

:  6 . 00 

Median 

7.000 

Median 

:  53 . 00 

Median:200. 1 

Mean 

:  6 . 13 

Mean 

6.928 

Mean 

:  52 . 75 

Mean:201.2 

3rd  Qu. 

:  7 . 00 

3rd  Qu. 

9.000 

3rd  Qu. 

:  57 . 00 

3rd  Qu. :220 . 6 

Max. 

:  9 . 00 

Max. 

9.000 

Max. 

:  97 . 83 

Max. : 304 . 0 

RACE 

MAJOR 

ETHNIC 

SOURCE 

GPA  MATH 

Black: 

71 

0:1036 

Asian 

38 

NROTC: 1024 

:  12  99  : 

1299 

Other: 

81 

1:  247 

Hisp: 

89 

OCS :  289 

1:  252  1: 

185 

White: 1992 

2:  861 

None 

1349 

USNA:  831 

0:  593  0: 

660 

Other 

668 

PHYSICS  COLLEGE  PIPELINE  NASCRAW 

: 1299  0:837  NPH:733  Min.:  0.00 

1:  249  1:476  NPM:571  1st  Qu.:58.55 

0:  596  2:831  NPJ:360  Median:92.00 

NPA: 190  Mean : 80 . 05 

NPT:  138  3rd  Qu .  :'95.58 
NPE: 137  Max. : 99. 99 

(Other) :  15 

PAG  PFSS  PASS  PIPELINE 

Min.:  0.00  Min.:17.00  Min.: 17.0  helo:733 

1st  Qu.:  91.40  1st  Qu.:44.00  1st  Qu.:46.0  jet:703 

Median:  94.33  Median:51.00  Median:53.0  prop:708 

Mean:  92.63  Mean:50.83  Mean: 51. 9 

3rd  Qu.:  96.40  3rd  Qu.:57.00  3rd  Qu.:59.0 

Max.:100.00  Max.:99.00  Max.:95.0 


PFG 

Min . : 0 . 000 
1st  Qu . : 3 . 04 6 
Median: 3 . 069 
Mean: 3 . 055 
3rd  Qu . : 3 . 092 
Max. :3. 991 
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